Enhanced Balancing of Bias-Variance Tradeoff in Stochastic Estimation: A Minimax Perspective

In “Enhanced Balancing of Bias-Variance Tradeoff in Stochastic Estimation: A Minimax Perspective”, the authors study a framework to construct new classes stochastic estimators that can consistently beat existing benchmarks regardless key model parameter values. Oftentimes biased estimators, such as finite-difference black box gradient estimation, require selection tuning parameters balance bias and variance ultimately minimize overall errors. Unfortunately, this relies on knowledge is unknown priori thus leads ad hoc choices practice. The introduce notion called asymptotic minimax risk ratio, which designed compare against benchmarks, whose values less than one imply could asymptotically outperform value. Based this, an outperforming weighting scheme by explicitly analyzing ratio via tractable reformulation nonconvex optimization problem.